<html>
<head>
<title>Version 2.0 NML Configuration Files</title>
</head>
<body  BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#0000EE" VLINK="551A8B" ALINK="#FF0000">

<h1>The Version 2.0 NML Configuration File Format and Tool</h2>

<H2>Background</H2>

<P>The version 1 configuration file format described in <A HREF="NMLcfg.html">&quot;Writing NML Configuration Files&quot;</A> has remained stable since 1995 except for some rarely used options that can be placed at the end of buffer and process lines . That format is also the only format that NML constructors can 
currently use directly and it is the format generated by the  
<A HREF="nmlbuild.html">NML Configuration File Builder</a> (a 
CGI script and HTML form)  and by the <A HREF="diagapplet/rcsdesign.html">RCS Design Tool</A> (a Java application).</p>
<P>In May 1999, the configuration format described here was proposed. The goals for this new configuration format were the following:</p>
<UL>
<LI>Easier for multiple organizations to share configuration files.</LI>
<LI>Less verbose files made possible by allowing users to specify only items that need to be different than the default.</LI>
<LI>Fewer errors caused by novice users.</LI>
<LI>Eliminate some tedious tasks.</LI>
</UL>
<P>The version 2 configuration format is supported only through a tool that takes files written in the new format and produces files in the older format. These files can then be used in an application in the same way as files written by hand in the older format or generated by one of the graphical tools.</p>
<p>The reader for this document is expected to be familiar with either C++ or Java, the NML Programmer's Guide, <A HREf="NMLcpp.html">C++ Version</A>/<A HREF="NMLjava.html">Java Version</A>, and with the the older configuration file format.</p>


<H2>Configuration File Line Types</H2>

<P>The new configuration files are ASCII text files, that contain eight different types of lines that can be idenfied by the first character or word on the line:</P>



<P><TABLE BORDER=1>
     <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">Starting Word or Character</FONT>
      </TD><TD>
         <P>Line Description.
      </TD></TR>
   <TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">#</FONT>
      </TD><TD>
         <P>Comment Lines starting with a single '#' have no effect on the output file. Comment lines starting with &quot;##&quot; are also placed in the output file.
      </TD></TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">##</FONT>
      </TD><TD>
         <P>Insert lines have the same effect as regular comment
         lines except that they are inserted as comments in the
         output file.
      </TD></TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">include</FONT>
      </TD><TD>
         <P>Include lines cause the contents of another file to be
         read as if the text were included at that point in the
         original file.
      </TD></TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">define</FONT>
      </TD><TD>
         <P>Definition lines define a variable that can be used
         through the rest of the file, by preceding the variable name
         with &quot;<B>$</B>&quot;  and surrounding it with parentheses &quot;<B>( )</B>&quot; like this &quot;<B>$(</B><I>varname</I><B>)</B>&quot;
	.
      </TD>
   </TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">define</FONT>
      </TD><TD>
         <P>Definition lines define a variable that can be used
         through the rest of the file, by preceding the variable name
         with &quot;<B>$</B>&quot;  and surrounding it with parentheses &quot;<B>( )</B>&quot; like this &quot;<B>$(</B><I>varname</I><B>)</B>&quot;
	.
      </TD>
   </TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">format_source_pattern</FONT>
      </TD><TD>
         <P>
	   <strong>Update February-2009:</strong>
	   Sets a pattern that will be used  to determine the format_source from
	   the format name by replacing the first &quot;%s&quot; in the pattern
	   with the format_name.
      </TD>
   </TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">format_header_pattern</FONT>
      </TD><TD>
         <P>
	   <strong>Update February-2009:</strong>
	   Sets a pattern that will be used  to determine the header from
	   the format name by replacing the first &quot;%s&quot; in the pattern
	   with the format_name.
      </TD>
   </TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">buffer_default</FONT>
      </TD><TD>
         <P>Buffer Default lines set defaults that affect buffer
         lines that occur after this line.
      </TD></TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">b</FONT>
      </TD><TD>
         <P>Buffer lines begin with must contain "<B>name=</B>" and
         the name of the buffer somewhere on the line. These lines
         are used to create and entry for a particular buffer.
      </TD></TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">process_default</FONT>
      </TD><TD>
         <P>Process Default lines set defaults that affect process
         lines that occur after this line.
      </TD></TR>
   <TR>
      <TD VALIGN=top WIDTH="126">
         <P><FONT SIZE="+1">p</FONT>
      </TD><TD>
         <P>Process lines must contain either "<B>name=</B>" or
         "<B>bufname=</B>" somewhere on the line and are used to
         create an entry linking a particular process to a particular
         buffer. For proper checking process lines must occur after
         the buffer line for the buffer the process is connecting to.

      </TD></TR>
</TABLE></P>

<H2>Buffer Variables</H2>

<p>The following variables with the exception of &quot;name&quot; can be set either on a buffer line to modify only one buffer or on a buffer default line to affect several buffers:</p>

<DL>
<DT>name</DT>
<DD>The name used as an argument to the NML constructor in the C++  or Java code. It must be unique. There is no default value, this variable must be set on each line with a "b ".</DD>
<DT>buftype</DT>
<DD>The type of buffer to create which can be <b>shmem</b>,<b>globmem</b>,<b>filemem</b>,<b>locmem</b> or <b>phantom</b>. The default value is shmem.</DD>
<DT>host</DT>
<DD>The host name where a server must be run if any processes are going to connect remotely. The default value is localhost.</DD>
<DT>size</DT>
<DD>The size of the largest message that can be sent to the buffer. The amount of memory allocated will be slightly larger than this to accomodate some handshaking flags. The default value is 960.</DD>
<DT>neutral</DT>
<DD>Whether the local buffer should be neutrally encoded. Buffers should be neutrally encoded if they can be accessed by multiple CPU types by a Bit3 adaptor for example or to force messages to go through format and update functions that significantly reduces message size such as when variable length arrays are used. Messages are always neutrally encoded when sent over a network. The default value is false.</DD>
<DT>bufnumber</DT>
<DD>A unique number used to identify the buffer within a server. The default value is calculated based on the position of the buffer line in the file. If its default value is modified with a default buffer line the default value for subsequent lines will still be incremented from this starting value.</DD>
<DT>max_proc</DT>
<DD>The maximum number of processes that can connect to a buffer locally. It does not affect remote processes or shmem buffers using the default mutual exclusion mechanism. The default value is calculated based on the number of processes connecting to this buffer.
<strong>Update February 2005:</strong> The number is ignored unless GLOBMEM or
mutex=mao is specified.
</DD>
<DT>key</DT>
<DD>A unique number used to identify the shared memory  and semaphore used for mutual exclusion in a shared memory buffer. It is relavent when using the &quot;ipcs&quot; or &quot;ipcrm&quot; commands. The default value is calculated based on the position of the buffer line in the file.</DD>
<DT>bsem</DT>
<DD>A unique number used to identify the semaphore used for blocking reads. Ther default value is -1, so blocking reads are not allowed by default. However if the value is changed with a default buffer line then subsequent lines will increment this starting value.</DD>
<DT>vme_addr</DT>
<DD>The VME address used for GLOBMEM on a VME backplane. The default value is 0 which is unusable. However if the default value is changed with a default buffer line then subsequent buffers will use the sum of this values plus the size of the preceding buffers.</DD>
<DT>remotetype</DT>
<DD>The protocol that should be used by remote processes connecting to this buffer which could be tcp,stcp, or udp. The default value is tcp.</DD>
<DT>port</DT>
<DD>The TCP or UDP port used by remote processes. The default value is 30000.</DD>
<DT>enc</DT>
<DD>The nuetral encoding method which can be xdr,ascii,or disp. The default value is xdr.</DD>
<DT>queue</DT>
<DD>Whether messages in this buffer should be queued. Setting it to a value greater than 1 also multiplies the size of the buffer by this value.The default value is 0.</DD>
<DT>diag</DT>
<DD>Whether to enable supplemental timing diagnostic information to be logged to the buffer. ( See <A HREF="timingdiag.html">Supplementary NML Timing Diagnostics Tools</a>.)</DD>
<DT>format_name</DT>
<DD><strong>Update February-2009:</strong> 
  Sets the name of the format function that should be used with the buffer. The format name should not include the final &quot;_format&quot; If the NML constructor should have been passed ex_format as the first argument, the tag in the nml file &quot;format_name=ex&quot; would accept this. If the format name does not match the process will print an error message, NML::valid() will return false, and  NML::error_type will be set to NML_FORMAT_NAME_DOES_NOT_MATCH_ERROR. If format_source_pattern or format_header_pattern is used then the format_name and either format_source or header was not set explicitly the format_name will be used to determine them. If the size is not set explicitly it can be determined by searching for the comment with the line &quot;Estimated_size  MAXIMUM&quot; followed by a size to use for the buffer in the generated source file. &quot;format=&quot; is also accepted as a shorter verion of &quot;format_name=&quot;.
</DD>
<DT>format_source</DT>
<DD><strong>Update February-2009:</strong> 
  Sets the name of the C++ file with the autogenerated format and update functions that should be used with the buffer. If the size is not set explicitly it can be determined by searching for the comment with the line &quot;Estimated_size  MAXIMUM&quot; followed by a size to use for the buffer in the generated format source file.
</DD>
<DT>header</DT>
<DD><strong>Update February-2009:</strong> 
  Sets the name of the C++ header file with the definitions of the message classes that should be used with the buffer. If neither the format_source nor format_name is set explicitly the format source will be assumed to be the base of the deader name plus &quot;_n.cc&quot;.  If the size is not set explicitly it can be determined by searching for the comment with the line &quot;Estimated_size  MAXIMUM&quot; followed by a size to use for the buffer in the generated format source file.
</DD>
</DL>


<P>There are some additional flags that were available in the old configuration file that the new tool will not recognize. This will produce a wargning but the old flag should simply be pasted to the end of the generated buffer line which should allow the use of the unrecognized flag.</p>


<H2>Process Variables</H2>

<P>The following variables can be place either on a process line to affect only one process connection to one buffer or on a default process line to affect multiple connections:</p>

<DL>
<DT>name</DT>
<DD>The name of the process that is passed to the NML constructor in the C++ or Java constructor. There is no default value, this variable must be set.
<strong>Update February.2005:</strong> The special value of &quot;default&quot; will match any name. The paremeters on this line affect any process not specifically mentioned earlier in the file. It is different than setting a process_default parameter in that setting a process default affects all processes subsequently named while this typically affects processes not mentioned in the file at all.Setting both the process name and buffer name to default matches any combination.
</DD>
<DT>bufname</DT>
<DD>The name of the buffer that this process is connecting to. There is no default value, this variable must be set.
<strong>Update February.2005:</strong> The special value of &quot;default&quot; will match any buffer name. The paremeters on this line affect any buffer for this process not specifically mentioned earlier in the file. It is different than setting a process_default parameter in that setting a process default affects all buffers subsequently named while this typically affects buffers not mentioned in the file at all. Setting both the process name and buffer name to default matches any combination.
</DD>
<DT>proctype</DT>
<DD>The type of process which can be local, remote or auto. Local processes use some form of shared memory directly, remote processes go through TCP, or UDP and require a server. Processes set to auto will attempt to determine if direct shared memory access is possible by comparing IP addresses.<strong>Update February 2005</Strong>
The default is auto.</DD>
<DT>host</DT>
<DD>The name of the host this process is running on. It is currently only used to comment the output file. The default value is localhost.</DD>
<DT>ops<DT>
<DD>The operations allowed by this process on this buffer which can be <b>r</b>, <b>w</b>,or <b>rw</b>. (For READ_ONLY, WRITE_ONLY, and READ_WRITE respectively) The default value is rw.</DD>
<DT>timeout</DT>
<DD>The time in seconds to allow before this process should timeout waiting for a read or write specified as a double or &quot;<B>INF</B>&quot; to indicate infinity.The default value is infinity.</DD>
<DT>master</DT>
<DD>Whether this process will be the master of this buffer. The master creates and clears the buffer when it is started. The default value is false.</DD>
<DT>server</DT>
<DD>Whether this process will act as a server for this buffer. The value of 2 has the special meaning that the process will spawn a server but then continue to access the buffer locally. The default value is false or 0.</DD>
<DT>c_num</DT>
<DD>The connection number which should be unique among processes connecting to the same buffer locally. The default value is calculated based on the number of processes that connected to this buffer before.
<strong>Update February 2005:</strong> The number is ignored unless GLOBMEM or
mutex=mao is specified.
</DD>
<DT>sub</DT>
<DD>The subscription interval in seconds for remote processes. The default value is -1 which indicates no subscription.</DD>
</DL>

<H2>Tool Command Line Arguments</H2>

<P>The program &quot;nmlcfg&quot; converts files from the new format to the old format which must be done because the NML constructors can not currently read the new format.</P>

<P>All command line not preceded by one of the following flags
is considered to be input files that should be in the new format. If more
than one input file is specified they are read in the order they occur and only one output file will be produced.</p>

<P>Here are the additional flags:</p>

<DL>
<DT>-D name=value</DT>
<DD>Defines a value on the command line just as if a define line was used.</DD>
<DT>-I directory</DT>
<DD>Adds a directory to a list of directories to be searched if a include line is found.</DD>
<DT>-o output_file</DT>
<DD>Specified the file where the old format file will be written.</DD>
</DL>

<H2>Examples</H2>

<P>The following example includes a separate file that can be modified to 
change the host name, 2 buffers, and 3 processes connecting to it.</p>

<P>The file <a href="myhosts">myhosts</a> contains the following.</p>

<PRE>
# Set host aliases.
define host1=localhost
</PRE>

<P>The file <a href="newnmlcfgex.cfg">newnmlcfgex.cfg</a> 
  contains the following:</p>

<PRE>
include myhosts

# Set the pattern used to find the header file by:
# Replace %s with the value from format_name to find find the main
# C++ header file associated with the buffer that prototypes the format function.
format_header_pattern %s.hh

# Set the pattern used to find Estimated_Sizes and format 
# function implementation.
# Replace %s with the value from format_name to find the main
# C++ file associated with the buffer with the autogenerated format function
# implementation.
# if the next line is not recognized you need atleast version 
format_source_pattern %s_n.cc

buffer_default host=$(host1)
b name=ex_cmd format_name=exCmd
b name=ex_stat format_name=exStat

process_default server=1 master=1 proctype=local name=mysvr
p bufname=ex_cmd
p bufname=ex_stat

process_default server=0 master=0

process_default name=pl
p bufname=ex_cmd
p bufname=ex_stat


process_default name=ex
p bufname=ex_cmd
p bufname=ex_stat

## Double pound comment get preserved in output config file.
## Special default process line that will be used by any process that does not match above.
p bufname=default name=default proctype=auto server=0 master=0

</PRE>

<P>The old style/output config file generated is this <a href="newnmlcfgex.nml">newnmlcfgex.nml</a>.</P>

<PRE>
# newnmlcfgex.nml
 

# Buffers:
#       name 	   type  	      host 	size 	neut 	0 	buf# 	max_proc 	. . .
B     ex_cmd 	SHMEM    	     localhost 	128 	0 	* 	1 	* 	35001 	TCP=20001  format_name=exCmd format_source=exCmd_n.cc header=exCmd.hh packed
B    ex_stat 	SHMEM    	     localhost 	256 	0 	* 	2 	* 	35002 	TCP=20002  format_name=exStat format_source=exStat_n.cc header=exStat.hh packed
 

# Processes: 
#       Name 	    Buffer 	      type 	      host 	       ops 	server 	timeout 	master 	c_num  	 . . .
P      mysvr 	    ex_cmd 	     LOCAL 	 localhost 	        RW 	1 	INF     	1 	0 	
P      mysvr 	   ex_stat 	     LOCAL 	 localhost 	        RW 	1 	INF     	1 	0 	
 
P         pl 	    ex_cmd 	     LOCAL 	 localhost 	        RW 	0 	INF     	0 	1 	
P         pl 	   ex_stat 	     LOCAL 	 localhost 	        RW 	0 	INF     	0 	1 	
 
P         ex 	    ex_cmd 	     LOCAL 	 localhost 	        RW 	0 	INF     	0 	2 	
P         ex 	   ex_stat 	     LOCAL 	 localhost 	        RW 	0 	INF     	0 	2 	

# Double pound comment get preserved in output config file.
# Special default process line that will be used by any process that does not match above.
P    default 	   default 	      AUTO 	 localhost 	        RW 	0 	INF     	0 	0 	
</PRE>

<P>The files <a href="exCmd.hh">exCmd.hh</a>,<a href="exCmd_n.cc">exCmd_n.cc</a>, <a href="exStat.hh">exStat.hh</a> and <a href="exStat_n.cc">exStat_n.cc</a> also had an effect on the output.</p>

<HR>

<p>Last Modified: February-2009</P>
<P>If you have questions or comments regarding this page or you would like to be notified of changes to the RCS library via email, please
contact  <A HREF="http://www.isd.cme.nist.gov/staff/shackleford/"
>Will Shackleford</A> at <I><A HREF="mailto:shackle@cme.nist.gov">shackle@cme.nist.gov</A></I></P>

</body>
</html>

